Everything about Gradient Descent totally explained
Gradient descent is an
optimization algorithm. To find a
local minimum of a function using gradient descent, one takes steps proportional to the
negative of the
gradient (or the approximate gradient) of the function at the current point. If instead one takes steps proportional to the gradient, one approaches a
local maximum of that function; the procedure is then known as
gradient ascent.
Gradient descent is also known as
steepest descent, or the
method of steepest descent. When known as the latter, gradient descent shouldn't be confused with the
method of steepest descent for approximating integrals.
Description
Gradient descent is based on the observation that if the real-valued function
